Statistical properties of spike trains: universal and stimulus-dependent aspects 
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Statistical properties of spike trains measured from a sensory neuron tn-vivo are studied experi- 
mentally and theoretically. Experiments are performed on an identified neuron in the visual system 
of the blowfly. It is shown that the spike trains exhibit universal behavior over short time, modulated 
by a stimulus-dependent envelope over long time. A model of the neuron as a nonlinear oscillator 
driven by noise and an external stimulus, is suggested to account for these results. The model 
enables a theoretic distinction of the effects of internal neuronal properties from effects of external 
stimulus properties, and their identification in the measured spike trains. The universal regime is 
characterized by one dimensionless parameter, representing the internal degree of irregularity, which 
is determined both by the sensitivity of the neuron and by the properties of the noise. The envelope 
is related in a simple way to properties of the input stimulus as seen through nonlinearity of the 
neural response. Explicit formulas are derived for different statistical properties in both the univer- 
sal and the stimulus-dependent regimes. These formulas are in very good agreement with the data 
in both regimes. 



I. INTRODUCTION 



Many cells in the nervous system respond to stimula- 
tion by generating action potentials (spikes). Time se- 
quences of these spikes are the basis for encoding infor- 
mation and for communication between neurons [n|. A 
pattern of spikes across time contains, in addition to the 
message being encoded, the signature of the biophysical 
spike generation mechanism, and of the noise in the neu- 
ron and its environment |y|. These factors are generally 
inter-related in a complicated way, and it is not clear how 
to disentangle their effect on the measured spike train. 

The biophysical mechanism for generating action po- 
tentials was first described successfully by Hodgkin and 
Huxley [0. Their description accounted for the stereo- 
typed shape of an action potential, which is a robust 
property, largely independent of external conditions. The 
Hodgkin Huxley model describes the neuron as a complex 
dynamical system; sustained firing (a continuous train of 
spikes) comes about when the dynamical system is driven 
into an oscillatory mode. This picture is consistent with 
experiments on isolated neurons: many of these tend to 
fire periodic spike trains in response to direct current 
injection, implying an oscillator like behavior. The fre- 
quency of these trains is deterministically related to the 
strength of the applied current; Adrian (1928) suggested 
long ago that this property could be used to code the 
strength of the input. Different neurons vary in the shape 
of the response function relating frequency to input. Fol- 
lowing Hodgkin and Huxley, many microscopic models of 
the neuron were constructed in the same spirit [^. One 
aim of this type of modeling is to produce the different 
frequency-current (///) response curves by fitting model 
parameters. 

Spike trains measured in vivo, however, show a very 
different behavior: many neurons seem to fire stochas- 
tically, even when external conditions are held fixed. 



This fact initiated what seems to be an unrelated line 
of research, that of describing spike trains by models of 
stochastic processes Ma. These models can sometimes 
describe correctly statistical properties of the spikes 
trains, such as the distribution of intervals, but in gen- 
eral the parameters of the models remain unrelated to 
physiological characteristics of real systems H] . 

Several fundamental questions concerning the statis- 
tical properties of spike trains thus remain unresolved, 
despite the large literature on this subject: How is the 
periodic behavior of the isolated neuron to be reconciled 
with the more irregular behavior in a complex network? 
What is a useful characterization of the degree of this 
irregularity, and how does it depend on external condi- 
tions? How sensitive are the statistical properties to the 
microscopic biophysical details of the neuron, and to the 
statistics of the noise? Can effects of the sensory stimulus 
be separated and recognized at the output? 

Here we present a theory which provides some answers 
to the above questions. We use the notion of a frequency 
function to describe the neuron's response S, and con- 
nect it to the stochastic firing in a network through the 
introduction of noise. Under some conditions we find that 
the statistical properties of spike trains are universal on 
the time scale of a few spikes [g[ . This means that they 
are independent of the details of the internal oscillator, of 
the noise and of external stimulation. All these are cap- 
tured by a single dimensionless parameter, related to the 
phase diffusion of the oscillator; this parameter charac- 
terizes the internal irregularity of the point process. On 
the time scale of many spikes, the universal behavior is 
modulated by an envelope reflecting the input stimulus. 
We present experimental data for the statistical proper- 
ties of spikes trains measured from an identified motion 
sensor in the visual system of the blowfly, under various 
external stimulation. These data are shown to be very 
well described by the theory. 



The paper is organized as follows: In section 2 we de- 
fine our model, and show how the frequency of the oscil- 
lator is related to the rate of the measured point process. 
In Section 3 we consider the statistical properties of the 
model, when the fluctuations in the inputs are rapid rel- 
ative to the typical interspike time l/r. We derive ex- 
plicit formulas for the statistical properties of the spike 
train, and show that all details affect these properties 
only through the irregularity parameter. In Section 4, 
we consider the case of an additional slow time scale in 
the inputs, which is much longer than l/r. We show that 
the conditional rate can be approximated by a product 
of two distinct parts: a universal part, depending only 
on the irregularity parameter, and a stimulus-dependent 
part, which modulates it. In each section a comparison 
of the theoretic results with measurements from the fly 
is presented. 



II. FREQUENCY INTEGRATOR 

A sequence of spikes will be described as a train of 
Dirac (S-functions: 



p{t) = J2sit-tk). 



(1) 



In this approximation the height and shape of the action 
potentials are neglected, and all the information is con- 
tained in their arrival times: the spike train is a point 
process. If the system is driven by a signal s{t) and a 
noise n{t), both continuous functions of time, then we 
can imagine that the neuron evaluates some functional 
J-'[s{t),n{t)] and produces a spike when this crosses a 
threshold: 



Pit) 



J2s(Hsit'),n{t')]-k\ ^. 



(2) 



Formula (||) describes a very general class of models: a 
spike is generated when J^ crosses a fixed threshold, and 
the process resets after spiking. The operator J^ can be 
linear or nonlinear, deterministic or stochastic, and can 
depend on the history of the signal and the noise in a 
complicated way |1^. We focus on the following more 
specific form of J-': 

T[s{t),n{t)]= f f[s{u)+n{u)] du = $(t), (3) 



where / > is the frequency response function charac- 
terizing the neuron. This model is related to Integral 
Frequency Pulse Modulation models and to the stan- 
dard integrate-and-fire model |llUlQ,M|. The motivation 
for defining a deterministic frequency response function 
comes from the measured behavior of isolated neurons in 
response to direct current injection. When driven by a 



constant stimulus s, in the absence of noise, our model 
neuron generates a periodic spike train with a frequency 
/(s), consistent with the behavior in isolation. Starting 
from a microscopic level of modeling, many parameters 
need to be tuned to produce a required form of the /// 
relation |13]. Here we use the /// relation as a phe- 
nomenological description of the neuron, and base the 
statistical theory on this description. 

Now we would like to "embed" our model neuron in a 
noisy environment, such as a complex sensory network, 
while it is still subject to a constant stimulus s. In gen- 
eral there can be many noise sources in such a network: 
the signals coming in from the external world are not 
perfect, the sensory apparatus (such as photoreceptors) 
is noisy, connections between cells in the network intro- 
duce noise, and finally the cell itself can generate noise 
(for example, channel noise |lj]). We introduce noise as 
an additional random function n{t) added to the input. 
This simplified scheme is justified by the fact that final 
results do not depend on the details of the noise distri- 
bution, therefore n{t) is understood as an effective noise. 

Retaining the notion of a local frequency, the noise 
causes frequency modulations in the spike train. Under 
the conditions s = const, n^O the oscillatory behavior is 
related to some periodic trajectory in parameter space. 
Assuming that this trajectory is stable, the addition of 
n{t) will cause the system to occupy a volume in param- 
eter space surrounding this trajectory. The phase of the 
oscillator will not advance at a constant rate /(s), but 
instead will be given by 



m = f[s+n{t)]. 



(4) 



The strength of the noise and the sensitivity of /(•) to 
changes in the inputs, both determine the frequency mod- 
ulation depth, or the amount of randomness in the phase 
advancement. 

The frequency function is an internal property of the 
neuron. One would like to relate it to the firing rate 
function in the presence of noise, which can be measured 
experimentally. Considering still the case of a constant s 
and introducing the average over noise (•••), the average 
spike train is 



k \ I 

Using the Poisson summation formula, 

^S{X - k) =Y^ e'^Trmo. 
fe m 

we can write the average spike train as 

{pM) = E (e''^'"*(*)<i>(i)). 



(5) 



(6) 



(7) 



For a stationary noise n{t), and time long enough for 
the system to be in a steady state, this average is inde- 
pendent of the time t. We assume that the noise has a 
short correlation time r„; this implies that the noise can 
have an arbitrary distribution at each time, but that it is 
uncorrelated with the noise value at a time much later. 
For t larger than the noise correlation time, t„, $(i) is 
an integral of many independent random variables, and 
is approximately Gaussian by the central limit theorem. 
Then, one can substitute the average of the exponent by 
the exponent of the two first cumulants: 



J2TTm<I>{t)\ ^ i27rm(*(t))-27r^m^(i5*(t)^) 



(8) 



where ((5$(t)^) = ($(i)^) - {'^(t))^- The first two cumu- 
lants of <I>(i) are: 






{f[s+n{t')]dt') = rt 



/ dt'dt"{df[s+n{t')]df[s+n{t")]). 
Jo 



(9) 



But / is correlated only over a short time, on the order of 
T„ and therefore the double integral can be approximated 
by 

(<5$(i)') « fdij^ dCiSfis + n{i+^-))Sfis + n{i~^))) 



Dt, 



(10) 



with D = Tn{5p). Using Eq. (|) in (0), and taking the 
time t arbitrarily large, we find 



(p(s)) = {f{s + n)) = [f*G]{s), 



(11) 



where * denotes convolution and G is the distribution 
function of the noise at a given point of time. For a sta- 
tionary noise distribution, this implies that the average 
firing rate as a function of signal s has the form of /(s) 
smeared by the noise. This result is independent of the 
exact form of the function / or of the noise distribution. 
It relates the /// curve to a measurable quantity, an av- 
erage firing rate in the presence of noise; in the measured 
quantity, much of the fine details of / will be smeared by 
the noise. 

In order to compare to data, we must now identify the 
stimulus s(t) in the experiment. If the neuron is isolated 
in a dish and current is injected directly into the cell, 
then s{t) should be naturally identified with this cur- 
rent. In experiments with an intact sensory system one 
would like to connect s(t) with the external stimulus. In 
the following discussion we will use as our test case the 
visual system of the blowfiy. In our experiment, a live 
immobilized fiy views various visual stimuli, chosen to 
excite the response of the cell HI. This large neuron is lo- 
cated several layers back from the eyes, and receives input 
through connections to many other cells. It is identified 



as a motion detector, responding optimally to wide-field 
rigid horizontal motion, with strong direction selectivity 
pl| , p2[ . The fly watches a screen with a random pattern 
of vertical dark and light bars, moving horizontally with 
a velocity s{t). We record the electric signal of HI extra- 
cellular ly, and register a sequence of spike timings {tk} 

il- 

An advantage of this system is that we have empiri- 
cal knowledge of what stimulus feature is relevant to this 
cell: it responds to wide field motion in the horizontal 
direction. Thus we may identify the input to the cell 
directly with a one dimensional external signal, the mo- 
tion of the pattern on the screen. Figure 1 shows the 
firing rate averaged over many presentations of the same 
stimulus, both as a function of time (Fig. la) and as a 
function of the instantaneous value of s{t) (Fig. lb). If 
the velocity on the screen is slowly varying in time, the 
firing rate of HI follows this velocity closely: Fig. la 
shows the time-dependent firing rate r(t) = {p[s{t)]) of 
HI in response to a random signal. Since the signal varies 
very slowly, one may use Eq. (O) locally, and if we plot 
the firing rate as a function of the instantaneous value of 
s, we see the smoothed response function /(s) (Fig. lb). 
If the signal varies rapidly, filtering mechanisms at early 
stages of the visual pathway become important, and s(t) 
which directly drives the cell is a modified version of the 
velocity on the screen. In this case, it is more difficult 
to map out directly the response curve |24| . It should be 
noted, however, that we present the response function in 
Fig. 1 only for purpose of illustration; in what follows 
we shall assume that this function is well defined, but 
we will not need to know its exact shape. Moreover, our 
results will show that to account for the statistical prop- 
erties of the spike trains very little information about the 
response function is required. 
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FIG. 1. Firing rate of HI as a function of time, averaged 
over trials: r{t) — {p{t)} [dots], compared to the input signal 
s{t) [solid line], for a slow signal (a) and a fast signal (c). We 
repeat the signal many times to obtain a sampling of the noise 
ensemble. The units of velocity are spacings of the compound 
eye's lattice (ommatidia) per second. 



It should be noted that the response function of the 
neuron /(•) is not a fixed property, but may change with 
external conditions. The theory presented here is valid 
within a steady state, in which /(•) takes a particular 
form and does not change with time. Rather than being 
a limitation, the context-dependence of /(•) opens the 
possibility to investigate adaptive changes in the neural 
response, when adjusting to different steady states p4|. 



III. THE UNIVERSAL REGIME 

In this section we present the statistical theory for the 
case of a constant input signal, s{t) = s, and a random 
short correlated noise of arbitrary distribution. The re- 
sult is a renewal process with special symmetry prop- 
erties, reflecting the underlying neuronal oscillator. We 
provide explicit formulas for the correlation function and 
the number variance. As will be shown in later sections, 
the results obtained here are valid in a limited time range 
if the stimulus s{t) is time dependent. 

A. Distribution of Intervals 

The distribution of inter-spike intervals is the distri- 
bution of times for which <l>(t) = 1. Due to the unidirec- 
tionality of the phase diffusion, these times are unique, 
and so the probability density is simply 



Pit) 



Sit-^-\l)) 



smt)~iMt))). (12) 



It is more convenient to calculate the cumulative distri- 
bution. 



Fit) 



Pit')dt' 



(13) 



which can be expressed in terms of the step function, Q, 
and its Fourier transform: 



Fit) = {QiHt) 1)) 



"'P ~i2TTp I i2TTp<^{t)\ 



2Trip 



(14) 



Using the Gaussian approximation (g) for ^(i), we find 
that 



Fit) 



dp 
2T:ip 



J2TTprt-2Tr^p^Dt 



- 1 + erf , 
2 V \V2Di 



and the interval density is 



P{t) = 



rt + 1 
VSirDt^ 



,-(rt-l)V2Dt 



(15) 



(16) 



In Appendix 1 this result is derived for a discrete sum of 
non-negative random variables, directly from the central 
limit theorem. The density (nfl) depends on two parame- 
ters, the average firing rate r and the diffusion coefficient 
D, both of dimensionality [time]~^. In the derivation, we 
used only the non-negativity of the frequency integrator 
to write down Eq. (I2h, and the Gaussian approxima- 
tion (@) for ^(t); therefore changes in the model which 
retain these properties will not affect the interval den- 
sity. Eq. (n6|) is similar to the first passage time of the 
Wiener process 0,^11: it has the same exponent, but this 
exponent is multiplied by a different function of t. As 
will be shown below, this results in significantly different 
symmetry properties of the function. 

To understand the qualitative properties of the point 
process, it is convenient to examine it in dimensionless 
time units, namely to define the time such that the av- 
erage rate is 1. We denote this time as x — rt. In this 
representation, the statistics depend on one dimension- 
less parameter, 7, 



_ D _ 

r 



if) 



and the interval density is 



Pix) = 



x + 1 



-(x-l)^/2tx 



(17) 



(18) 



The parameter 7 governs the decay of the interval den- 
sity both near the origin and at large t. For small 
7 these decays are strong, indicating a narrow density, 
and for large 7 they are weaker and the distribution is 
broader. More formally, the moment generating func- 
tion, G(A) = (e"'*'^), of (Esl) can be calculated using the 
integral representation. 



P(.t) = / dpil + iiT^p)' 



^i2'rT(x — l)p—27r fp x 



The result is 



G(A) 



1 



1 



1 



eT 



2 V' ■ \/l + 2A7 
and the first two moments are 

(.t) = 1 + 7/2 
((5a;2) =7+47'- 



i(l-yi+2A^) 



(19) 



(20) 



(21) 
(22) 



Note that the average interval length is not equal to the 
inverse of the average rate, which is 1 in our units. In 
general, inversion does not commute with averaging; for 
small 7, however, the inverse average and the average of 
the inverse are similar. 



Defining the coefficient of variation by y/ {6x^) / (x) 
p4| , it is seen that the coefficient of variation is approxi- 
mately proportional to .y/7 for small 7, and to 7 for large 
7. It can take on arbitrarily small and large values de- 
pendent upon 7. This is a more quantitative way of see- 
ing that our family of distributions interpolates between 
low-variability (or regular) and high-variability (or ir- 
regular) limits. It should be noted that this distribution 
arises from a simple integration model of many indepen- 
dent inputs pq|. As noted already by several authors 
[p^,p[ , it is not only the properties of the inputs but also 
of the internal neural response that determine the degree 
of irregularity of a spike train. In our model, the param- 
eter controlling this irregularity (|l^), accounts for both 
these effects. 

The fact that 7 controls the behavior of the density 
( |l8| ) at both tails, takes the quantitative form of an in- 
variance under the transformation x —f l/x, with the 
Jacobian properly accounted for. This implies that the 
cumulative distribution of the intervals between succes- 
sive spikes is identical to the distribution of inverse in- 
tervals, when both are measured in dimensionless units. 
Since our spike train is a renewal process, this invariance 
of the interval distribution implies an invariance of the 
process as a whole. In general, for a point process with 
time intervals {xi}, one may construct the dual process 
with intervals {1/xi}; this has the natural interpreta- 
tion of local frequencies. The process defined by (Q) is 
self-dual: all statistical properties of the dual process are 
identical to the original one. For the process defined by 
(O) , this invariance holds up to a global rescaling of the 
axis. Fig. 2 shows that this is indeed a property of the 
measured data: the cumulative distributions of intervals 
and inverse intervals overlap when plotted in dimension- 
less units. 



exactly k—1 other spikes in between them. In their land- 
mark paper, Gerstein and Mandelbrot (1964) observed 
that the scaled interval distributions of low orders in spike 
trains from the cat cochlear nucleus, have a similar shape. 
This observation motivated them to suggest the random 
walk model for the membrane voltage. In our model, we 
can calculate the scaled interval distribution directly: it 
is the distribution of times for which $(<) = k, 



ft(() 



^2" 



erf 



rt — k 



(23) 



In this notation, the interval distribution of Eq. (Ill) 
is the scaled distribution of order 1. Figure 3a shows 
the first three scaled interval distributions, as measured 
experimentally, together with Eq. (^3|). The two fitting 
parameters of the theory, the average rate r and the diffu- 
sion constant D, are fitted once for all three graphs. Ac- 
cording to the observation of Gerstein and Mandelbrot, 
these curves should have the same shape after rescal- 
ing the time axis to dimensionless units rt/k. Figure 3b 
shows the scaled interval distributions in dimensionless 
time units. These curves have a similar shape, but do not 
quite overlap. As is easily seen from Eq. (p3|), the trans- 
formation t -^ t/k gives a function of the same general 
form, but with a different value of D. Thus the family 
of curves Fk (t) with parameters r, D obey the following 
equation: 
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(24) 
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FIG. 2. A symmetry of the spike train point process: 
the distribution of intervals between neighboring spikes is the 
same as the distribution of the inverse intervals (local frequen- 
cies), up to a constant scaling factor. Data are shown from 
two experiments with constant velocity stimuli, of magnitude 
10.5 (a) and 0.16 (b) deg/s. 



where the superscripts denote the dependence on the pa- 
rameters. In Figure 3b, it is clearly seen that the steep- 
ness of the curve increases with increasing k, consistent 
with a decrease in the diffusion constant D. The point 
rt/k = 1 is where the distributions cross, in agreement 
with the theoretical equation (p4|). 
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A generalization of the interval distribution is the 
scaled interval distribution of order k, defined as the dis- 
tribution of intervals between pairs of spikes that have 



FIG. 3. (a) Scaled interval distributions of orders 

k = 1,2,3, representing the probability of finding a pair of 
spikes at a given time interval, with exactly k — 1 spikes in 
between. Data are shown in grey dotted lines, while theory 
(Eq. E3) is shown in a solid black line. Fitting parameters of 
the theory, r and D, are fitted once for all three curves, (b) 
Scaled interval distributions in dimensionless time units. In 
agreement with Eq. (|24|), these curves are similar but have 
slightly different slopes, corresponding to an effective value 
of D which depends on k. The curves cross at rt/k — 1, as 
predicted by Eq. (|^. 

The derivation of the interval density is independent 
of many microscopic details of the neuron and of the 
noise, and it is therefore expected to describe correctly 
the behavior of many different systems. Figure 4 shows 
a comparison of the interval distribution ( |l6| ) with ex- 
perimental data from different parts of the visual system 
in several animals. Figure 4a shows data measured in 
our experiment on the motion sensitive neuron HI in the 
visual system of the blowfly. In this experiment, the fly 
watched a random pattern of dark and light bars moving 
at a constant velocity of ~ 0.16 deg/sec. The best fit 
to the data was found with an irregularity parameter of 
7 = 0.1. Figure 4b is adapted from data published by 
Robson and Troy (1987). In this experiment, a station- 
ary sinusoidal grating was presented to an anesthetized 
cat, and spikes were recorded from neurons in the retina. 
The particular neuron these data were recorded from was 
identified as a "Q-type" neuron, characterized by regu- 
lar spiking. The best fit of Eq. ( p^ ) was obtained with 
7 = 0.015. Figure 4c shows data measured from isolated 
goldfish retina by Levine and Shefner (1977) in dark- 
ness. These data are well described by Eq. ( [l6| ) with 
7 = 0.1. Figure 4d contains data measured by Cattaneo 
et al. (1981) from visual cortex of anesthetized cat, in 
response to a drifting sinusoidal grating. This interval 
density is best fit with 7 = 0.3. 
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FIG. 4. Distribution of intervals between neighboring 
spikes, experiments and theory. The data are from difi^er- 
ent sensory systems in different animals: (a) fly visual system 
(lobula plate motion detector), (b) Cat retina, (c) goldfish 
retina (d) cat visual cortex. For more details about these pub- 
lished data see text. The y-axis is the probability of seeing 
an interval, in units of 1/s (a-c), and the relative probability 
in percent (d). The solid black lines are a fit of Eq. (jlq), and 
number in each panel indicates the value of the irregularity 
parameter 7 in the fit. 

These data indicate that the classification of spike 
trains to universality classes according to the irregular- 
ity parameter 7 is a useful one, and can be applied to 
many systems. Systems with very different microscopic 
properties can belong to the same universality class, and 
their interval density is well described by a theory with 
one dimensionless parameter. It will be shown in sec- 
tion 4, that this classification can be applied also under 
conditions of rich dynamic stimuli; in this case, the in- 
ternal irregularity parameter 7 describes the statistical 
properties on short time scales. 



B. Correlation function 

An important statistical property of the spike train 
is its (auto)-correlation function, {p{t)p{Q)). Whereas 
many models have been used to calculate the interval 
density, less attention has been paid to the correlation 
function [ p6[ . In Appendix B we derive the correlation 
function under the assumption that the noise correlation 
time T„ is much shorter than the typical interval between 
spikes l/r. The result is: 



(p(t)p(O)) = rSit) + r 
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D)f 
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(25) 
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where P^' ' (t) are the densities derived from the scaled 
interval distributions of Eq. (p3|) , 



:,ir:D) 
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Jrt-kf/2Dt 



(26) 



It is convenient to think about the probability per unit 
time of finding a spike at time i ^ 0, conditional on the 
event that a spike is found at time i = 0. This quan- 
tity R(t) is proportional to the correlation function in 
the region t ^ 0, and is called the conditional rate: 



R{t) 



fc#0 






it). 



(27) 



The density labeled k is the probability per unit time 
for finding a pair of spikes separated by a time t with 
exactly (fc — 1) spikes in between. These independent 
events, when added together, give the total probability 



per unit time to find a pair of spikes separated by a time 
t, which is just the conditional rate. For small k these in- 

(r D) 

dividual densities P^ ' are narrow and their peaks can 
be resolved, and as A; — > oo they smear and overlap to give 
asymptotically R{t) ~ r. The degree of regularity, I/7, 
is associated with the number of densities which can be 
resolved. The notation Rjj' '(t) is introduced to empha- 
size that this is a universal function, independent of the 
detailed neuronal response /(s), and of the detailed prop- 
erties of the noise. Figure 5 shows the conditional rates 
for experiments with constant velocity, together with the 
best fit to Eq. ( p5| ) . The constant value of the velocity 
stimulus s increased among parts (a-d) of the figure; the 
average firing rate r increases and the irregularity 7 de- 
creases. This is clearly due to refractory effects: as the 
firing rate increases, the repulsive interaction among the 
spikes becomes more important, and the spike train be- 
comes more stiff, causing a more regular behavior. The 
form of the repulsive interaction and the function describ- 
ing the correlation function, however, remain the same. 
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FIG. 5. Conditional rate for the spike train in experiments 
with a constant velocity stimulus. Data are shown by a gray 
histogram, calculated directly from the spike times by bin- 
ning them into 4 ms bins. Theoretical expression (Eq. GSf) is 
shown by a solid black line. The four parts of the figure cor- 
respond to different values of the constant velocity stimulus; 
0.7 (a), 2.6 (b), 10.5 (c) and 42.2 (d) deg/sec. As the motion 
signal becomes stronger, the average firing rate increases and 
the irregularity parameter decreases (see also Fig. 6). 



C. Number variance 

The number variable N(t) is a random function which 
counts the number of spikes in the time window [0 : t\. 
Defined by 
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p{t')dt', 



(28) 



it provides a useful, less detailed, characterization of the 
spike train point process. To study the statistics of this 
variable, we write it as: 



N{t) = ^emt) - k) =Intmt)) 



(29) 
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where Int(?/) is the integer part of y. Its Fourier repre- 
sentation is 



m = m + l-^+j:^ 
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where ip is uniformly distributed on [0,1], accounting 
for a random position of the first spike in a window 
(see App. C). The number mean is {N{t)) = rt, and 
around this mean N{t) fluctuates. Since the correla- 
tion function of spikes is of finite range, the long time 
asymptotic behavior of the number variance is diffusive: 
alf — {SN'^{t)) ^ Dt ^. It is of interest, however, to de- 
rive a complete expression for this quantity also for short 
times, where correlations between spikes are important. 
Using (BG), it is shown in Appendix C that 
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Written as function of the number mean, i.e. as a func- 
tion of the dimensionless time variable x — rt, the number 
variance is: 
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Figure 6 shows the number variance as a function of the 
number mean, as calculated from an experiments with 
constant velocity stimuli. Part (a) shows a scatter-plot 
of the values obtained in various windows in the experi- 
ments, with a line showing the average. Part (b) of the 
figure shows a detailed view of one such average plot, 
with the theoretical prediction Eq. ( p^ ) in solid black 
line. The number variance of a Poisson process is shown 
for comparison. The "diffusion constant" in the number 
variable is 7, implying again its role as an irregularity pa- 
rameter: the more stochastic the point process, the faster 
is the diffusion in the number variance. Similar to Fig- 
ure 4, a higher constant signal induces a higher firing rate 
and a lower irregularity of the spike train. Although the 
data presented here have 7 < 1, this is not a fundamen- 
tal property of the theory, and in general, 7 can take on 
also values larger than 1, resulting in a "super-Poisson" 
behavior. 
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FIG. 6. Number variance as a function of number mean. 
(a) Scatter plots of variances, as calculated in different time 
windows in the experiment. Data are shown from two ex- 
periments, with constant velocities of 0.054 and 0.162 deg/s. 
Solid lines indicate the average value of the variance for a 
given mean, (b) Comparison to theory: the average value at 
each number mean, is compared to the theoretic formula (Eq. 
B4), which is shown by a solid black line. In this experiment, 
the velocity was 0.162 deg/s. 
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FIG. 7. Dimensionless irregularity parameter, 7, as a 
function of the average firing rate r. The different symbols 
correspond to two different experiments, performed each with 
a different set of constant velocity signals. 



IV. TIME DEPENDENT STIMULUS 



D. Irregularity and stiffness of spike trains 



We have presented a statistical theory for a frequency 
integrator model under constant stimulation and short- 
range correlated noise. This results in a renewal point 
process, with the density of intervals given by Eq. (16). 
This process is characterized by an irregularity parame- 
ter 7, which depends both on the variance of the noise 
and on the sensitivity of the frequency response to noise; 
it is the depth of frequency modulation, resulting from 
these two effects, that determines the irregularity of the 
process. This one-parameter family of processes can de- 
scribe many different spike trains, ranging from almost 
periodic to almost Poisson. 

In the data presented above, a correlation was found 
between the global average firing rate and the irregular- 
ity 7. Figure 7 shows a plot of the different values of 
7 obtained by fitting to the equations in this section, 
for two sets of experiments. The irregularity decreases 
roughly linearly with the firing rate, with a saturation at 
very high firing rate. This is the limit where the abso- 
lute refractory period is approached. The simple relation 
between 7 and r holds only for the case of constant stim- 
uli, where frequency modulations are essentially induced 
by noise. When these modulations are affected also by 
a time varying sensory stimulus, very different behaviors 
is found (see next section). 



In the previous section, we considered the case of a 
constant input signal, s{t) = s. More generally signals 
coming into the system have a temporal structure, and 
additional time scales enter the problem. If s{t) is slowly 
varying relative to 1/r, universal behavior is expected 
on short times, and slower modulations will appear on 
longer times. In this section we consider the statistical 
properties in the case of a slowly varying input signal. We 
show that the conditional rate can be approximated by a 
product of the universal function Eq. (^^ , and a slowly 
varying envelope reflecting the temporal correlations of 
the input signal. This envelope is calculated for some 
simple cases. It is shown that the theory fits the data 
very well, even when the time scale separation required 
in theory is only marginally satisfied by the experimental 
conditions. 



A. The Telegraph Approximation 

Consider a random input signal s(t), that can take on 
positive as well as negative values. Figure 8 shows a seg- 
ment of some spike trains recorded from HI in response 
to such a stimulus, which was repeated many times. The 
most striking effect in the figure is the existence of wide 
regions with spikes, and wide regions which are empty; 
the typical time for these regions is much larger than the 
interspike time. This partition into regions is a conse- 
quence of the direction selectivity of the HI neuron: it 
fires when the effective stimulus is in its preferred di- 
rection, and is inhibited by stimulus in the opposite di- 
rection. Although the velocity stimulus one the screen 
varies rapidly in this experiment (every 2ms an indepen- 
dent value is chosen) , the spiking and quiet regions in the 



spike trains have a much slower typical time scale. This 
results from intermediate processing: the velocity on the 
screen is not identical to the effective stimulus driving the 
cell, since it is filtered by the photoreceptors and other 
elements in the visual pathway. 

To describe the phenomenon of spiking regions and 
quiet regions, we imagine the spike train to be multiplied 
by a telegraph signal, which keeps track of the algebraic 
sign of the effective stimulus: 
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/(i)p^W, 



(33) 



where p^{t) — Q[s{t)] is the telegraphic envelope of the 
spike train. Figure 8 shows an illustration of the tele- 
graph signal, which demonstrates that the the time scale 
of the effective stimulus sign change is longer than the 
typical spike time, 1/r. 
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FIG. 8. Response of the HI neuron to repeated presen- 
tations of a time-dependent stimulus. Each line shows the 
response to one of the presentations, with the origin t — 
corresponding to the onset of the stimulus; each dot repre- 
sents a spike . The black solid line is an illustration of the 
telegraphic envelope that multiplies the spike train, due to 
the strong direction selectivity of the neuron. 

Assuming that the telegraph envelope is statistically 
independent of the short time structure in the spike train, 
one may write the correlation function as a product. 



{p(t)pm 



(p^(i)p^(0))(p^(i)p^(0)). 



(34) 



Now brackets (• • •) denote averaging over both the noise 
and the random stimulus. To perform the averages, we 
use the separation of time scales: imagine dividing the 
time axis into blocks of size r^, the correlation time of 
the effective stimulus. Performing first the average over 
the noise in each block separately, the first term in the 
product gives the correlation function of Eq. (Ea), with 
the parameters r, D determined by the local value of s: 
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for t j^ 0. If the response is saturated, r{s) « rQ{s), and 
the firing rate does not change much inside each spiking 
region; moreover the diffusion constant D is determined 
mainly by properties the noise, and is the same in all the 
spiking regions. Therefore, on averaging the first term 
over s, one has aRjj' , where a is the coverage fraction, 
defined as the fraction of time in which the stimulus is 
positive {{p^{t)) = a). 

The second term in the product (B3) is a correlation 
function of the input signal as seen through the tele- 
graphic envelope: 



{p^{t)p^m = {Q[sit)]e[sio)]). 



(36) 



Special care should be taken around the point t = 0, 
since it is affected by the delta function singularities. The 
whole correlation function then takes the form 

(p(f)p(O)) « raS{t) + rRp''\t) (e[s(i)]e[s(0)]), (37) 



and the conditional rate is 



Rit) = R^t} > iQHt)Mm])- 



(38) 



This formula expresses the conditional rate as a product 
of two terms. The first term reflects internal properties 
of the noise and of the neuron, similar to the result of the 
previous section; Rjj is parameterized by an effective rate 
r and diffusion constant D. This function has an oscil- 
latory structure on a time scale 1/ D, which thus defines 
the universal regime of the correlation function, in which 
properties of the inputs do not have an important effect. 
The second term contains information about statistics 
of the incoming stimulus, as seen through the nonlinear 
response of the neuron. It modulates the universal func- 
tion with a slower structure. Intuitively, the condition of 
time scale separation can be understood as follows: on 
short times, t < 1/D, the envelope is almost constant 
and the oscillations of the universal part are visible. As 
these oscillations decay, on times t > 1/D, the stimulus 
induced structure sets in. The independence between the 
two factors affecting the probability of finding a spike at 
time t given a spike at time 0, results in a product form. 
Eq. (pq) indicates that in the telegraph approxima- 
tion, the envelope of the correlation function depends on 
the statistics of the zero-crossings of the stimulus. These 
statistics for a random continuous function are in general 
very difficult to calculate U% . Here we need only the cor- 
relation function of the algebraic sign of a random signal, 
and we can proceed in two ways. In the next sub-section 
we consider the case in which zero crossings are indepen- 
dent, and the resulting correlation function is a simple 
exponential. Another simplification occurs if the incom- 
ing stimulus has Gaussian statistics, and in this case one 
may derive an exact formula for the correlation function 
of the nonlinear rate. 



B. Random independent spiking regions 

Let us first assume that the spiking regions occupy 
random independent positions along the time axis, with 
lengths A drawn independently from some distribution. 
This is a crude approximation that may not be justified 
for many experimental conditions. However, it is the sim- 
plest type of telegraphic signal, and is characterized by 
a small number of parameters; therefore we use this ap- 
proximation as a starting point. The correlation function 
of such a telegraph signal is derived in Appendix D, and 
the result is, 
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(39) 



where p(A) is the distribution of lengths of positive re- 
gions in the telegraph signal, and /i is the average length 
of such a positive region. The second term clearly decays 
for {t — t') ^oo for a well behaved p(A), and asymptoti- 
cally there remains only the square of the average signal. 
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For the case of an exponential distribution 



of positive regions. 



p(A) = -e 



-a/m 



the correlation function is also exponential, 

(p^(i)p^(t'))-a' + aMe"^*~*'^/''- 



(40) 



(41) 



The exponential decay of the envelope, Eq. (p]), gives a 
good fit to many of the measured data sets. It has addi- 
tional fitting parameters, a and /i, which are the coverage 
fraction and average length of the positive regions in the 
telegraph envelope of the spike train. It uses no prior 
knowledge of the input stimulus, and relies primarily on 
the direction selectivity of the response. Figure 9 shows 
the correlation function calculated from spike trains, in 
two experiments with random signals, of 20IIz (a,b) and 
500Hz (c,d) bandwidth. Although in these experiment 
we know the properties of the visual stimulus presented 
to the fly, this stimulus is rapid, and pre-processing takes 
place in earlier stages of the visual pathway; the effective 
signal entering the HI neuron is therefore a filtered and 
probably distorted version of the motion on the screen. 
Therefore, we try the simple picture of the telegraph ap- 
proximation, rather than rely on the detailed properties 
of the visual stimulus. The black solid line in Figure 9 is 
a fit to a product of the universal function and the en- 
velope correlation in the telegraph approximation, (41). 
The time scale of the exponential decay is /i w 50ms for 
the slower varying stimulus, and fi ~ 20ms for the faster 
varying one. In the case of the fast stimulus, the cor- 
relation is probably limited by the filtering processes in 
the visual system, thus indicating that the time scale for 
these processes is « 20ms. This is on the order of the 



behavioral time scale for changes in flight course in these 
flies, found by Land and CoUett (1974) to be « 30ms. 




-(b) 


M 



(d) 

■ J 


(v : 



-20 20 

t (ms) 



-100 100 

t (ms) 



FIG. 9. Conditional rates for spike trains, in the short 
time and long time regimes. Data (gray histogram) are mea- 
sured in experiments where the visual stimulus was a pattern 
of light and dark bars moving with a random time-dependent 
velocity. The velocity signal has different band widths, of 20 
Hz (a,b) and 500 Hz (c,d). The theoretic formula of Eq. (^) 
is presented as a solid black line. 



C. A Gaussian input signal 

If the incoming random signal is drawn from a Gaus- 
sian distribution, one may calculate exactly the correla- 
tion function of a nonlinear response r{s). We focus on 
the simple case where the response is a step function at 
zero, and the stimulus has zero mean, corresponding to 
a coverage fraction of a = 1/2. In this case. 
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i(|c(t)|), 



(42) 



where c{t) = (s(t)s(O)) is the correlation function of the 
Gaussian signal. In principle, one may calculate the cor- 
relation function of a general nonlinear response r{s) in 
the Gaussian case, but for our purposes it is sufficient to 
consider the step response. We expect that in an experi- 
ment where the driving stimulus varies slowly, the effect 
of intermediate filters will be negligible and one can take 
the motion on the screen to be essentially equivalent to 
the stimulus s{t) driving the neuron. Figure 10 shows the 
correlation function measured from an experiment where 
the input signal was a slowly varying random function 
of time, with a Gaussian distribution. The correlation 
function as calculated from the data is shown as a gray 
histogram, whereas the solid black line is given by 



R{r) = rRp""^ 



I + 2:^arcsin(|c(i)|) 



(43) 
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where c(i) was taken from the known distribution of the 
input signal. The two fitting parameters are r, related 
to the global firing rate, and D, the diffusion constant in 
the universal regime. 




(b) 


K. : 



-200 




t (ms) 



200 



FIG. 10. Conditional rate for spike trains, in the short 
time and long time regimes. Data (gray histogram) are mea- 
sured in experiments where the visual stimulus was a pattern 
of light and dark bars moving with a random time-dependent 
velocity, which is very slow and has a Gaussian distribution. 
The theoretic formula of Eq. (^3|) is presented as a solid black 
line. 



D. Linear rectifier approximation 

In analogy to Eq. (|37|), one expects that if the non- 
linear response of the neuron is /(■), the conditional rate 
will take the form 
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(44) 



where r{s) is the firing rate as a function of stimulus, 
obtained by averaging over noise only, and r, D are some 
effective global parameters representing an average over 
different regimes where s is almost constant. In this sec- 
tion we show results from experiments where the response 
r{s) cannot be approximated by a step function, and 
apart from the direction selectivity the firing seems to 
follow the input stimulus linearly. A better approxima- 
tion for the response would therefore be a linear rectifier. 
We used sine wave stimuli of different amplitudes and pe- 
riods. Figure 11 shows the firing rate averaged over noise, 
together with the stimulus, for one such experiment. 
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FIG. 11. Stimulus and average response for a sine wave 
experiment. A random pattern of dark and light horizontal 
lines was moved by a sine wave velocity (solid black line) , and 
this stimulus was repeatedly presented to the fly. The average 
firing rate was calculated over the repeated presentations as 
a function of time (circles). The response of the neuron is not 
saturated, and it follows closely the positive part of th sine 
wave stimulus. (Compare the saturated response in Fig. la). 



We used the approximation 

r{s) « s0(s) 



(45) 



to evaluate the envelope of the conditional rate 
(r[s(i)]r[s(0)]). For a sine wave stimulus of frequency 
ri, a straightforward calculation yields 



{r[s{tMsm) = ^cos{il\t\) 
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(46) 



The conditional rate is, then, given by Eq. (p4[), with 
Eq. (Eq) as the envelope. If we use our knowledge about 
the frequency of the input signal, we expect to get a de- 
scription of the conditional rate with only the universal 
fitting parameters r and D, and no additional parameters 
for the envelope. Figure 12 shows two correlation func- 
tions for the sine wave experiments, in the universal (a,c) 
and the stimulus-dependent (b,d) regimes. No additional 
parameters of the stimulus other than its frequency were 
used; the solid black curve was obtained from Eq. (H) 



with the envelope given by (46). The periodic nature 



of the envelope is evident in the data, and the theoretic 
prediction describes both the short range and the long 
range features well. 
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FIG. 12. Conditional rates for spike trains in the short 
time and long time regimes. Data (gray histograms) are mea- 
sured in an experiment where a pattern of dark and light bars 
were moved horizontally with a sine wave velocity, with pe- 
riods of 0.25 sec (a,b) and 0.5 sec (c,d). Theory (black solid 
lines) is obtained from Eq. (p4[), with the fitting parameters 
r and D, and using the known frequency of the stimulus. 
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E. Irregularity and stimulus properties 



In this section, we showed that the conditional rate of 
the spike trains has approximately a product form. One 
term in the product is very similar to the conditional 
rate for a constant stimulus, and describes the behavior 
on short times. This term is universal in the sense that it 
depends on only two simple parameters, the global aver- 
age rate and the internal irregularity of firing. The other 
term carries information about temporal correlations in 
the stimulus, and is a long-time envelope over the uni- 
versal term. We have shown how this envelope can be 
calculated in several cases, giving a very good fit of the 
data. 




FIG. 13. correlation functions from different experiments 
with different time-varying stimuli, in dimensionless units. 
The overlap at short times indicates that the parameter left 
after rescaling, the internal irregularity, is similar in all these 
cases. 



A naive quasi static application of the universal theory 
would tell us that the short-time behavior is character- 
ized by local values of the parameters r, D, and that these 
change slowly as the external stimulus varies slowly. This 
seems inconsistent with the short time behavior exhib- 
ited by the data (Figures 8a, 10a, 10c). If the parame- 
ters would change with the local changes of the external 
stimulus, the oscillatory structure defined by a period of 
r would be considerably washed out when averaged over 
the different values of s. The data, however, show pro- 
nounced oscillations with a well defined period. This is 
consistent with the notion that in a changing environ- 
ment, the system achieves a steady state with the distri- 
bution as a whole, thus obtaining global values for the 
statistical parameters of its firing. 

Further evidence for this picture is given by Figure 13, 
where the correlation function is plotted in normalized 
time units, and is normalized by the height of the first 
peak. After rescaling of the two axes, there is only one di- 
mensional parameter describing the short time behavior: 
the irregularity 7. Although the correlation functions 
were measured under different experimental conditions of 
time varying stimuli, the curves are approximately over- 
lapping in the short time regime {x < 2). Thus, under 
high signal-to-noise ratio, where the frequency modula- 
tions are mainly induced by the stimulus and not by the 
noise, the system tends to fix the internal irregularity 
parameter 7 at some preferred value. This cannot be 
explained by a simple quasi static behavior; it proba- 
bly involves subtle adaptive mechanisms of the neuron 
which try to maximize the dynamic range in each stim- 
ulus ensemble. This subject is currently under further 
investigation ||24[] . 



V. DISCUSSION 

The statistical properties of spike trains generated by 
a sensory neuron under various stimulation conditions 
were considered. Experiments were performed in vivo on 
a blowfly, where the visual stimulus was well controlled, 
and spike trains were measured extracellularly from an 
identified motion sensor. This neuron is known to re- 
spond to wide field horizontal motion. 

The main theoretical questions addressed were: (i) how 
does the statistical behavior of spike trains in vivo relate 
to the biophysics of the spike generation mechanism in 
the cell, and (ii) how can the effects of internal properties 
be separated from those induced by the external sensory 
stimulus (or the input to the neuron). These questions 
were addressed within the framework of a model, which 
describes the neuron as a nonlinear oscillator driven both 
by noise and by an external stimulus. In general the effect 
of these two is the same: to cause frequency modulations 
in the oscillator. We considered here the case where the 
noise and the stimulus are very different in their temporal 
characteristics, namely the noise is rapid and the stim- 
ulus is slow, compared to the time scales typical of the 
spiking. This separation of time scales enables the the- 
oretic analysis of the model, which in turn provides an 
understanding of how the different factors are reflected 
in the statistical properties of the spike trains. 

It was found that on the time scale of a few spikes, sta- 
tistical behavior is rather universal and can be described 
by a renewal process. This process is derived from phase 
diffusion of a nonlinear oscillator, and has special sym- 
metry properties reflecting the underlying oscillator. It is 
characterized by one dimensionless parameter which rep- 
resents the internal degree of irregularity of the process. 
This parameter interpolates between highly regular and 



12 



highly stochastic Hmits. The parametrizaion allows spike 
trains from different parts of the visual system, and even 
from different organisms, to be classified in a simple way. 

On the time scale of many spikes, effects of the sensory 
stimulus become important and are reflected in the form 
of a slowly varying envelope which modulates the univer- 
sal functions. Using only simple features of the nonlinear 
neuronal response, the theory provides quantitative pre- 
dictions for the statistical properties, which are found to 
be in very good agreement with our measured data in 
both the short time and the long time regimes. 

Justification for assuming a separation of time scales 
between noise and stimulus comes from the fact that sig- 
nals in the visual system are filtered, and therefore the ef- 
fective signals reaching an interneuron are relatively slow. 
As often is the case in comparing theory and experiment, 
we found that the agreement of the theoretical predic- 
tions with the data extend to a regime where the required 
separation of time scales is only marginally satisfied by 
the experimental conditions. 

The understanding of how the stimulus is reflected 
in the statistical properties of the spike train could be 
used "backwards" : in cases where little is known about 
the stimulus, analysis of the long time behavior of the 
correlation function can give us information about the 
time scales involved in this stimulus, with only a gross 
description of the neural response (direction selectivity, 
degree of saturation). Possibly this understanding can be 
extended to cross-correlation between several neurons; in 
that case the separation between stimulus-induced and 
internal properties will be important in assessing the 
connections between the neurons. This is subject for 
future research. 



Appendix A 

In this Appendix we show that for a unidirectional ran- 
dom walk, (a "thermal ratchet" walk), the density of 
times between barrier crossing is given by Eq. (18), 
in the limit where many steps are needed to cross the 
barrier. 

We consider the discrete case. Let Xi be non-negative 
random variables (xi > 0), identically distributed and 



independent, with (x) = fi and {6x'^) = a^ . Deflne the 
random variable of their sum as 

^jv = ^x,. (A.l) 

1=1 

Let C be a constant positive number, then for large 
enough TV one has from the central limit theorem. 



where erf (x) = f /n e~* dt. Now define N\ to be the 
number of steps required to first reach the barrier. Then 
from the non-negativity it follows that 

ProbjXAT <C}^ ProbjA^i > iV} = 1 - Prob{7Vi < iV}. 

(A.4) 

There are some normalization subtleties here, but let us 
imagine that the sample space is composed of trajecto- 
ries with a finite number of steps M which is much larger 
than the typical number needed to cross threshold. Then, 
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Appendix B 

In this Appendix, we derive Eq. 
the definition, 
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As expected for a point process of average rate r. For 
times t y^ 0, we calculate the conditional rate R{t), the 
probability of firing at some time t given a spike at time 
0: 
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Using the integral representation of the Theta function. 
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As before, we use the Gaussian approximation for $(i), 
Eq. (g), which is justified for times t satisfying r„ <C t: 
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where v — rt — k. Doing first the integral dp, we find 
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Appendix C 

Here we discuss the properties of the number variable, 
which counts the number of spikes in the window [0 : t\. 
One must specify how the point i = is chosen, and 
there are two natural choices: (i) start counting at a 
spike, and (ii) start counting at a random point in the 
spike train. The second choice of a random origin is 
the more commonly used. In this case, it is convenient 
to define an auxiliary variable ip = $(ii), the phase of 
the integrator $(i) at the time of the first spike in the 
window. This variable is uniformly distributed in [0 : 1], 
and with this we have 

N{t) =Int[$(t) + l-(^] 

oo ^ 

^<Ut) + l-ip+ V sm\2Trm(m) + l-ip]- -. (C.l) 

'^-^ nm 2 

?n— 1 

Note that the constant — ^ ensures that A^(0) = 0. Since 
if only depends on the choice of origin, it is independent 
of $(i), and therefore 
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= rt+ y 



m^O 



2Trim 
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= rt. 



(C.2) 



Now to calculate the number variance we define the fluc- 
tuation 



9 < ^ 9777 7 



2 ^ 2TTim, 



(C.3) 



and average its square: 
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By Eq. (^, ((5$(i)^) = Dt; averaging the second term 
over ip gives j^. Due to the independence of ip and $(i), 
the first cross-term vanishes while the second cross-term 
decouples into 



2 y^ / \2Trim\'S>(t)+l]\l -2Tvimv\ 
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Performing the average over ip, 
d 1 



(C.5) 



{ipe 



—i27rm^\ 



— 2'Kinnp^\ 



dfi 2711171 



/^l 



-1 

2T:im ' 



(C.6) 



and we have for the second cross-term 

oo -. 

y _L^ cos(2^mrt)e-2-'™'^*, 



(C.7) 



where the Gaussian approximation was used when aver- 
aging over $(i). In the last double sum of Eq. (C^), all 
terms vanish by averaging over p except for the terms 
m + m' = 0, which gives 



^(2-H^ 12 
Adding the terms together we find 



(C.8) 
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which is the same as Eq. (plf). 



Appendix D 

In this Appendix we derive Eq. ( pl| ) for the correlation 
function of the envelope of the spike train, in the tele- 
graph approximation. The envelope p^{t) is composed 
of a train of characteristic window functions, 
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(p"(i)> = Ex(r^-^'^'»+^) 



(D.l) 



as illustrated in Fig. 14. This function has a Fourier 
transform 



-j^T,. sin(a;Afc/2) 



E' 



k 



ico/2) 



+ aS{u), (D.2) 



where T^ denote the middle points of the positive signal 
regions. 



FIG. 14. 

These positive regions are assumed to be distributed 
over the time axis independently with a density (3 per 
unit time, and with an average length of (A) = /x. Aver- 
aging in the frequency domain gives 



(Ip'^MP)^/?' 



(sin2(ojA/2)) 



+ iPfif6{iu). (D.3) 



(c./2)2 
In the time domain this expression transform to 



(p^(i)p^(O)) = (/3/i)' 



(D.4) 



/? / p{A)dA 



dujsm'^{ujA/2) 
2^ (w/2)2 ' 



The last tern in this expression is the Fourier transform 
of a product of two sine function, which is the convolu- 
tion of two square windows. This convolution has the 
form 



X*X 



, |i| > A 
A-|tLk|< A 



(D.5) 



which is equivalent to Eq. (pa). 
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